Markov Decision Evolutionary Games with Expected Average Fitness

Aim: To model and characterize evolutionary games where individuals have states that are described by controlled Markov chains. The action of an individual in a local interaction with another randomly selected individual determines not only the instantaneous fitness but also its probability to move to another state. The goal of a player is to maximize its time average fitness. Mathematical methods: The main mathematical tool is occupation measures (expected frequencies of states and actions). This tool is a central one in the theory of Markov Decision Processes. We make use of the geometric properties of the set of achievable occupation measures. Key assumption: Under any pure stationary policy of an individual, its Markov chain has a single ergodic class of states. Results: We define and characterize a new concept of Evolutionarily Stable Strategies based on the concept of Occupation Measures. We relate this set to the concept of Evolutionarily stable set (ESSet). We present a way to transform the new type of evolutionary games into standard ones. Applying this novel framework to energy control in wireless networks, we show existence of an Occupation Measure ESS (OMESS).